Search for: All records

Blindrestoration of low-quality faces in the real world has advanced rapidly in recent years. The rich and diverse priors encapsulated by pre-trained face GAN have demonstrated their effectiveness in reconstructing high-quality faces from low-quality observations in the real world. However, the modeling of degradation in real-world face images remains poorly understood, affecting the property of generalization of existing methods. Inspired by the success of pre-trained models and transformers in recent years, we propose to solve the problem of blind restoration by jointly exploiting their power for degradation and prior learning, respectively. On the one hand, we train a two-generator architecture for degradation learning to transfer the style of low-quality real-world faces to the high-resolution output of pre-trained StyleGAN. On the other hand, we present a hybrid architecture, called Skip-Transformer (ST), which combines transformer encoder modules with a pre-trained StyleGAN-based decoder using skip layers. Such a hybrid design is innovative in that it represents the first attempt to jointly exploit the global attention mechanism of the transformer and pre-trained StyleGAN-based generative facial priors. We have compared our DL-ST model with the latest three benchmarks for blind image restoration (DFDNet, PSFRGAN, and GFP-GAN). Our experimental results have shown that this work outperforms all other competing methods, both subjectively and objectively (as measured by the Fréchet Inception Distance and NIQE metrics). 

Clinical outcome or severity prediction from medical images has largely focused on learning representations from single-timepoint or snapshot scans. It has been shown that disease progression can be better characterized by temporal imaging. We therefore hypothesized that outcome predictions can be improved by utilizing the disease progression information from sequential images. We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images. In our method, a self-attention based Temporal Convolutional Network (TCN) is used to learn a representation that is most reflective of the disease trajectory. Meanwhile, a Vision Transformer is pretrained in a self-supervised fashion to extract features from single-timepoint images. The key contribution is to design a recalibration module that employs maximum mean discrepancy loss (MMD) to align distributions of the above two contextual representations. We train our system to predict clinical outcomes and severity grades from single-timepoint images. Experiments on chest and osteoarthritis radiography datasets demonstrate that our approach outperforms other state-of-the-art techniques. 

Intramolecular charge transfer and the associated changes in molecular structure in N,N′-dimethylpiperazine are tracked using femtosecond gas-phase X-ray scattering. The molecules are optically excited to the 3p state at 200 nm. Following rapid relaxation to the 3s state, distinct charge-localized and charge-delocalized species related by charge transfer are observed. The experiment determines the molecular structure of the two species, with the redistribution of electron density accounted for by a scattering correction factor. The initially dominant charge-localized state has a weakened carbon–carbon bond and reorients one methyl group compared with the ground state. Subsequent charge transfer to the charge-delocalized state elongates the carbon–carbon bond further, creating an extended 1.634 Å bond, and also reorients the second methyl group. At the same time, the bond lengths between the nitrogen and the ring-carbon atoms contract from an average of 1.505 to 1.465 Å. The experiment determines the overall charge transfer time constant for approaching the equilibrium between charge-localized and charge-delocalized species to 3.0 ps.